Multimodal One-Shot Learning of Speech and Images

Imagine a robot is shown new concepts visually together with spoken tags, e.g. "milk", "eggs", "butter". After seeing one paired audio-visual example per class, it is shown a new set of unseen instances of these objects, and asked to pick the "milk". Without receiving any hard labels, could it learn to match the new continuous speech input to the correct visual instance? Although unimodal one-shot learning has been studied, where one labelled example in a single modality is given per class, this example motivates multimodal one-shot learning. Our main contribution is to formally define this task, and to propose several baseline and advanced models. We use a dataset of paired spoken and visual digits to specifically investigate recent advances in Siamese convolutional neural networks. Our best Siamese model achieves twice the accuracy of a nearest neighbour model using pixel-distance over images and dynamic time warping over speech in 11-way cross-modal matching.Comment: 5 pages, 1 figure, 3 tables; accepted to ICASSP 201

Towards Learning to Speak and Hear Through Multi-Agent Communication over a Continuous Acoustic Channel

While multi-agent reinforcement learning has been used as an effective means to study emergent communication between agents, existing work has focused almost exclusively on communication with discrete symbols. Human communication often takes place (and emerged) over a continuous acoustic channel; human infants acquire language in large part through continuous signalling with their caregivers. We therefore ask: Are we able to observe emergent language between agents with a continuous communication channel trained through reinforcement learning? And if so, what is the impact of channel characteristics on the emerging language? We propose an environment and training methodology to serve as a means to carry out an initial exploration of these questions. We use a simple messaging environment where a "speaker" agent needs to convey a concept to a "listener". The Speaker is equipped with a vocoder that maps symbols to a continuous waveform, this is passed over a lossy continuous channel, and the Listener needs to map the continuous signal to the concept. Using deep Q-learning, we show that basic compositionality emerges in the learned language representations. We find that noise is essential in the communication channel when conveying unseen concept combinations. And we show that we can ground the emergent communication by introducing a caregiver predisposed to "hearing" or "speaking" English. Finally, we describe how our platform serves as a starting point for future work that uses a combination of deep reinforcement learning and multi-agent systems to study our questions of continuous signalling in language learning and emergence.Comment: 12 pages, 6 figures, 3 tables; under review as a conference paper at ICLR 202

Accelerating Online Reinforcement Learning via Supervisory Safety Systems

Deep reinforcement learning (DRL) is a promising method to learn control policies for robots only from demonstration and experience. To cover the whole dynamic behaviour of the robot, the DRL training is an active exploration process typically derived in simulation environments. Although this simulation training is cheap and fast, applying DRL algorithms to real-world settings is difficult. If agents are trained until they perform safely in simulation, transferring them to physical systems is difficult due to the sim-to-real gap caused by the difference between the simulation dynamics and the physical robot. In this paper, we present a method of online training a DRL agent to drive autonomously on a physical vehicle by using a model-based safety supervisor. Our solution uses a supervisory system to check if the action selected by the agent is safe or unsafe and ensure that a safe action is always implemented on the vehicle. With this, we can bypass the sim-to-real problem while training the DRL algorithm safely, quickly, and efficiently. We provide a variety of real-world experiments where we train online a small-scale, physical vehicle to drive autonomously with no prior simulation training. The evaluation results show that our method trains agents with improved sample efficiency while never crashing, and the trained agents demonstrate better driving performance than those trained in simulation.Comment: 7 Pages, 10 Figures, 1 Table. Submitted to 2023 IEEE International Conference on Robotics and Automation (ICRA 2023

Combinatorial nuclear level density by a Monte Carlo method

We present a new combinatorial method for the calculation of the nuclear level density. It is based on a Monte Carlo technique, in order to avoid a direct counting procedure which is generally impracticable for high-A nuclei. The Monte Carlo simulation, making use of the Metropolis sampling scheme, allows a computationally fast estimate of the level density for many fermion systems in large shell model spaces. We emphasize the advantages of this Monte Carlo approach, particularly concerning the prediction of the spin and parity distributions of the excited states, and compare our results with those derived from a traditional combinatorial or a statistical method. Such a Monte Carlo technique seems very promising to determine accurate level densities in a large energy range for nuclear reaction calculations.Comment: 30 pages, LaTex, 7 figures (6 Postscript figures included). Fig. 6 upon request to the autho

Extended Thromboprophylaxis with Betrixaban in Acutely Ill Medical Patients

Background Patients with acute medical illnesses are at prolonged risk for venous thrombosis. However, the appropriate duration of thromboprophylaxis remains unknown. Methods Patients who were hospitalized for acute medical illnesses were randomly assigned to receive subcutaneous enoxaparin (at a dose of 40 mg once daily) for 10±4 days plus oral betrixaban placebo for 35 to 42 days or subcutaneous enoxaparin placebo for 10±4 days plus oral betrixaban (at a dose of 80 mg once daily) for 35 to 42 days. We performed sequential analyses in three prespecified, progressively inclusive cohorts: patients with an elevated d-dimer level (cohort 1), patients with an elevated d-dimer level or an age of at least 75 years (cohort 2), and all the enrolled patients (overall population cohort). The statistical analysis plan specified that if the between-group difference in any analysis in this sequence was not significant, the other analyses would be considered exploratory. The primary efficacy outcome was a composite of asymptomatic proximal deep-vein thrombosis and symptomatic venous thromboembolism. The principal safety outcome was major bleeding. Results A total of 7513 patients underwent randomization. In cohort 1, the primary efficacy outcome occurred in 6.9% of patients receiving betrixaban and 8.5% receiving enoxaparin (relative risk in the betrixaban group, 0.81; 95% confidence interval [CI], 0.65 to 1.00; P=0.054). The rates were 5.6% and 7.1%, respectively (relative risk, 0.80; 95% CI, 0.66 to 0.98; P=0.03) in cohort 2 and 5.3% and 7.0% (relative risk, 0.76; 95% CI, 0.63 to 0.92; P=0.006) in the overall population. (The last two analyses were considered to be exploratory owing to the result in cohort 1.) In the overall population, major bleeding occurred in 0.7% of the betrixaban group and 0.6% of the enoxaparin group (relative risk, 1.19; 95% CI, 0.67 to 2.12; P=0.55). Conclusions Among acutely ill medical patients with an elevated d-dimer level, there was no significant difference between extended-duration betrixaban and a standard regimen of enoxaparin in the prespecified primary efficacy outcome. However, prespecified exploratory analyses provided evidence suggesting a benefit for betrixaban in the two larger cohorts. (Funded by Portola Pharmaceuticals; APEX ClinicalTrials.gov number, NCT01583218. opens in new tab.